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Abstract 

We study the dynamical states that emerge in a small- 
world network of recurrently coupled excitable neurons 
through both numerical and analytical methods. These 
dynamics depend in large part on the fraction of long- 
range connections or 'short-cuts' and the delay in the 
neuronal interactions. Persistent activity arises for a 
small fraction of 'short-cuts', while a transition to fail- 
ure occurs at a critical value of the 'short-cut' density. 
The persistent activity consists of multi-stable periodic 
attractors, the number of which is at least on the or- 
der of the number of neurons in the network. For long 
enough delays, network activity at high 'short-cut' den- 
sities is shown to exhibit exceedingly long chaotic tran- 
sients whose failure-times averaged over many network 
configurations follow a stretched exponential. We show 
how this functional form arises in the ensemble-averaged 
activity if each network realization has a characteristic 
failure-time which is exponentially distributed. 

Many systems in nature can be described as 
a network of interconnected nodes. A grow- 
ing list of examples, from social and ecological 
webs to the neural anatomy of simple organisms, 
have been shown to exhibit complex topological 
features in stark contrast to ordered lattices or 
purely random networks. It is futhermore be- 
coming increasingly clear that the details of the 
network architecture can crucially influence the 
emergent dynamics in the system. An under- 
standing of the dynamics on complex networks 
requires investigation into the interplay between 
the instrinsic dynamics of the elements at the 
nodes and the topology of the network in which 
they are embedded. Here we study the collec- 
tive behavior of excitable model neurons in a 
network with Small-world topology. The Small- 



world network is locally ordered but includes a 
certain number of randomly placed, potentially 
long-range connections which may act as short 
cuts. Such a topology bears a schematic re- 
semblence to that found in the cerebral cortex, 
in which neurons are most strongly coupled to 
nearby cells, but are known to make long-range 
projections to cells millimeters away. We show 
that in the regime of low 'short-cut' density, per- 
sistent activity arises in the form of propagating 
excitable waves which annihilate upon meeting 
and are spawned anew via re-injection by long- 
range connections. A critical value of the 'short- 
cut' density at which network activity fails can 
be found by matching the longest distance on the 
network, a topological measure, to an intrinsic 
recovery time of the individual cells. We fur- 
thermore show how activity once again emerges 
at high 'short-cut' densities if the delay in the 
neuronal interactions is sufficiently long. The ac- 
tivity in this regime exhibits long, chaotic tran- 
sients composed of noisy, large-amplitude popu- 
lation bursts. In this regime we investigate the 
interplay of the delay and the network topology 
numerically and provide a simple argument which 
provides a qualitative description of the distribu- 
tion of failure times of the chaotic network activ- 
ity. 



I. INTRODUCTION 

It has been widely recognized that the connectivity of 
a network of active elements has a profound impact on its 
function. Substantial effort has therefore been devoted 
to the characterization of the network connectivity []], 0| 
leading to the identification of various measures that are 
significant in determining the properties of the system. 
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Particularly relevant among them are the average and 
maximal length of paths connecting arbitrary nodes in 
the network and the distribution for the number of links 
(degree) that emanate from the nodes. 

The dynamics of networked elements has been stud- 
ied in particular detail for coupled oscillators, address- 
ing the influence of the network topology on their ability 
to synchronize. The existence of long-range connections 
between oscillators, which reduces the effective size of 
the network, has been found to enhance synchronizabil- 
ity substantially At the same time, the heterogeneity 
of the degree distribution of many such networks limits 
the ability of the oscillators to synchronize, requiring a 
balance between the two aspects Q. 

Excitable elements constitute a second important class 
of dynamical systems. In locally coupled networks they 
give rise to traveling waves (e.g. If these waves 

annihilate upon mutual collisions, which is typically the 
case, persistent activity usually requires an external drive 
or spontaneous excitation by noise. In models for neural 
systems driven by noise [1, 0, [l(| networks with non-local 
connections between their elements have been shown to 
possess the tendency to exhibit relatively ordered oscil- 
lations in the population (mean) activity. The spatial 
structure of such noise-induced waves becomes less co- 
herent with an increase in the non-local coupling [Til ] . 
Combining local connectivity with a small number of 
non-local connections allows a localized time-periodic ex- 
ternal input to entrain the whole system much faster than 
in a purely local network and at the same time the oscil- 
lations are much more coherent than in a truly random 
network El , [H[ . The dynamics depend very strongly on 
the duration of the refractory period relative to the prop- 
agation time scales and the range of the coupling. Thus, 
for very short refractory periods the small loops that are 
sufficiently likely in many types of random networks allow 
persistent activity even in the absence of noise as long as 
the initial conditions are asymmetric enough flij . In an 
epidemic model the introduction of non-local connections 
was found to induce a transition to a state with coherent 



population oscillations 15 1. 

In most studies the connections between the elements 
have been assumed to be bi-directional, which is a very 
natural assumption in an epidemic context [l5T ] and can 
be for neural systems if the neurons are coupled via gap 
junctions 0. In the absence of noise persistent activity is 
then only obtained with initial conditions that suitably 
break the reflection symmetry. If in an initially quies- 
cent network individual neurons are excited by an exter- 
nal perturbation the pair-wise excited symmetrical waves 
are, however, all eventually annnihilatcd in collisions. 

In neural systems the coupling between neurons is pre- 
dominantly not bi-directional; instead most connections 
are via chemical synapses in which usually the informa- 
tion is only transmitted from the pre-synaptic axon to the 
post-synaptic dendrite. For these situations it is appro- 
priate to consider directed networks with uni-directional 
connections. In previous work (l6j we have investigated 



networks in which the local connections are bi-directional 
assuming that due to the close phyiscal proximity the 
probability for reciprocal axo-dendritic connections is 
quite high, but the non-local connections ('short-cuts') 
are uni-directional. Considering this directed modifica- 
tion of the by now classical small- world network [l7j , we 
found that even a few such uni-directional short-cuts al- 
low for persistent activity even for generic localized ex- 
citations. As the density of short-cuts is increased, how- 
ever, more and more network configurations support only 
a brief population burst after which the activity dies out. 
For slow propagation speeds of the waves the failure of 
network activity was found to be delayed and could oc- 
cur after many cycles of chaotic population bursts. The 
simplicity of this model allowed an analytical result for 
the failure transition and detailed numerical analysis. 



The results for the minimal system [16| provided im- 
portant insight into the phenomena found in simlations 
of more elaborate models that were motivated by con- 
crete biological situations. In [l8[ the connection between 
the topology of a neural network and its tendency to- 
wards epileptic seizures has been studied and related to 
the degree of recurrent connectivity of different parts of 
hippocampus. The origin of bursting behavior was ad- 
dressed in 19]. Quite commonly bursting behavior arises 
when fast spiking behavior drives a slow process that in 
turn can shut off the spiking. Typically the slow pro- 
cess is associated with some slow kinetics. In the 
authors show that no such slow kinetics are needed in 
networks with small-world topology. Both the seizing 
and the bursting activity can be interpeted quite well 
based on an understanding of the failure transition in 
the basic model of [l6[ . The rapid spread of activity and 
the persistent oscillations have also been observed in re- 
cent experiments on the Belousov-Zhabotinsky reaction 
in which the uni-directional short-cuts were implemented 
exploiting the photo-sensitivity of the reaction [2(| HH • 



Here we build on our previous results and present 
a detailed characterization of the persistent states and 
the dependence of their properties on the prevalence of 
short-cuts as well as of the long chaotic transients for 
which we provide an understanding of the stretched ex- 
ponential behavior in their failure times. In Section |TT] 
we define the model, which consists of 'Integrate-and- 
Fire' neurons that are coupled via excitatory pulses in a 
small-world topology with uni-directional short-cuts. In 
Section rrri] we discuss in detail the persistent states and 
the cross-over from persistent activity to failure for the 
case of rapidly propagating waves. In Section UVl we pro- 
vide a detailed analysis of the exceedingly long chaotic 
transients in the regime of slower propagation. In the 
concluding Section [V] we discuss our results in light of 
other work on neural networks with small-world topol- 

ogy mtn. 
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II. NEURON MODEL AND NETWORK 
TOPOLOGY 

We consider a one-dimensional network of N identical 
integrate-and-fire neurons. Their membrane voltage is 
described by 



dt 



N 



-V + RI ext + 9syn 



with the reset condition 



V(t + ) — V res whenever 



V(t-) = V t tfi) 



Here r is the membrane time constant, g syn is the 
synaptic strength measuring the change in voltage due to 
one incoming action potential, Wij = or 1 indicates the 
absence or presence respectively of a synaptic connection 
from neuron j to neuron i, P is the time at which neuron 
j fires an action potential, I ex t is an external current and 
R the membrane resistance. The effective delay td in 
the neuronal interaction includes the time for the signal 
to propagate along the axon as well as the time needed 
for initiating the triggered action potential. If the latter 
time dominates over the axonal delay the dependence of 
the delay on the distance between the neurons involved 
can be neglected. Post-synaptic currents due to synaptic 
activation are considered instantaneous and are therefore 
modeled as Dirac-delta functions. Eq.([2]) reflects the fact 
that whenever the voltage of neuron i reaches the thresh- 
old value Vth, a spike is emitted and the voltage is reset to 
the value V res . We write EqQ] in terms of dimensionless 
quantities by setting without loss of generality V r e.s = 0, 
rescaling V and g syn by V res and time by r |3l| . and by 
replacing I ext by the steady-state voltage V^, to which 
it corresponds in the absence of synaptic input. We fo- 
cus on the case of excitable rather than spontaneously 
oscillating neurons, i.e. Voo < 1, and consider only initial 
conditions in which at most a few neurons are triggered to 
spike with all other neurons quiescent. In the absence of 
noise this implies that neurons can only fire at times that 
are multiples of td and the model can be solved exactly 
between these times. In the numerical computations the 
time step is therefore taken to be At = td- 

In cortex neurons receive often not only local input 
from neurons near-by but also input from some distant 
neurons through long-range projections. We mimic such 
a heterogeneous connectivity with an extremely simpli- 
fied architecture for the network in which each neuron is 
connected to 2k neighbors, i.e. tu,j = 1 for j < i ± k 
and j ^ i and an additional pN neurons that are cho- 
sen randomly. The parameter p thus gives the density of 
random uni-directional connections as a fraction of the 
total number of neurons N. 

The dynamics arising in Eqs (|ll2|) for purely local con- 
nectivity (p = 0) depend, in the absence of noise, on 
the interplay between the strength of the synapses g syn , 



the number of neighbors 2fc, and the delay td. This is 
most easily illustrated for nearest-neighbor connectivity 
(k = 1). In this case, if g syn + Voo < 1 the pre-synaptic 
input is insufficient to cause a spike and no network activ- 
ity will occur. For stronger input, i.e. for g syn + Voo > 1, 
a wave of speed 1/td is generated. After the spike the 
voltage is reset to V = and the neuron is only ready 
to fire again after it has recovered to the extent that an 
input of magnitude g syn is sufficient to trigger another 
spike. This recovery time is given by 



Tr = In 



Vac 



^oc H~ fjsyn 1 



(3) 



Note, that Tr is not an intrinsic refractory period of 
the neuron since for sufficiently strong input g syn this 
model neuron can fire arbitrarily fast. 

Due to the bidircctionality of the local connections the 
firing of each neuron not only triggers a spike in the neu- 
ron ahead of it in the wave of excitation, but gives also an 
input to the neuron behind it. Thus, the latter neuron re- 
ceives already a first input at a time 2tjj after its firing. If 
Tr > 2t£) this input is not sufficient to trigger a spike and 
the activity propagates away from the site of initiation 
as each neuron contributes exactly one spike, see FigfT^. 
If, however, Tr < 2to then the wave front entrains all 
the neurons in its wake, eventually leading to synchro- 
nized activity of the whole network. Unless autapses with 
wu =/= are included the network breaks up into two syn- 
chronous groups of neurons that fire out of phase with one 
another, see FigQJ>. In the general case, a propagating 

wave can be sustained if g syn e TD Yl n =i e ~ nTD + ^oo > 1- 
That is, a neuron receives inputs from k neighbors as 
the wave approaches, with an input at a distance n dis- 
counted by e^ 1- ") 1 " . For simplicity we will focus on the 
case of nearest-neighbor coupling (k = 1) , and will con- 
sider the regime in which waves of excitation propagate, 
but do not entrain activity in their wake. This choice 
constrains the allowable values of Voo and g syn - Specifi- 
cally, we take Voo = 0.85 and g syn = 0.2 unless otherwise 
noted. In this regime the collision of two waves leads 
to their mutual annihilation and after having fired in a 
propagating wave a neuron can be re-excited by a single 
input of size g syn after a time 



r« = in /r 



Qsyn 



i 



(4) 



This time includes the input after a delay of 2 tjj from 
the neuron further ahead in the wave. 

Eqs (jll2p with the above-mentioned assumptions are a 
minimal model for the generation and propagation of 
waves in cortical-like tissue. As we shall see, the ad- 
dition of random, long-range connections qualitatively 
alters the dynamics of the network, allowing for a rich 
variety of spatio-temporal patterns. 



4 



(a) 50 r 




2 3 

time 



FIG. 1: Dynamics in the flocal network of 50 neurons with 
nearest-neighbor coupling. Parameters are Voo = 0.85 and 
Td = 0.1. (a) g S yn = 0.2: An example of a propagating 
wave. The wave fronts meet and annihilate at time 2.5. Top: 
Raster plot showing the times at which the neurons fire an 
action potential. Middle: The averaged firing rate of the 
network. Bottom: The voltage of neuron number 20. (b) 
gsyn = 1.0: The wave entrains all the neurons in its path. 
The final state of the network is one in which two synchronous 
groups of neurons fire out of phase with one another. Note: 
The maximum possible firing rate is 1/td = 10. 



III. THE ORDERED REGIME: ATTRACTORS 
AND FAILURE 

Once the input current and synaptic strength g syn 
have been fixed, the dynamics arising in Eqns|T]) and ^ 
are determined by the remaining two parameters: the 
fraction of randomly placed connections p and the delay 
in the neuronal interaction td ■ 

The dynamics for non-vanishing p differ qualitatively 
from those without any long-range connections. The 
presence of long-range connections ('short-cuts') allows 
the waves of excitation to be re-injected into portions of 
the network which have already been excited. This pro- 
cess of re-injection may lead to persistent network activ- 
ity as shown in Fig[2j As the waves spread outward from 
the initial site of stimulation they encounter long-range 
connections and are injected elsewhere in the network. 
Wavefronts that meet annihilate. After some time, the 




************** 



« 5 wwwwwwvw 



I 'J. 



T7T77 

Hi;. 




60 

Time 



FIG. 2: Examples of the network dynamics as a func- 
tion of the density of short-cuts p. Here p = 
0.01,0.05,0.1,0.15,0.2,0.25 for (a) through (f) respectively. 
td = 0.1. 



activity settles into a stable pattern in which the rates of 
generation and annihilation of the waves balance. Aver- 
aged over time and across configurations, the firing rate 
of these persistent states increases rapidly with increas- 
ing number of short-cuts and saturates around p ~ 0.1 
(FigSJ. This saturation is a consequence of the neuron's 
finite recovery period T^ 1 ' given in Eq0] The bars in 
FigElgfve the standard deviation of the firing rate across 
configurations for td = 0.1. Clearly in some configu- 
rations the firing rate comes very close to its maximal 
value. As the system size is increased this saturation 
level is reached already for smaller values of p (FigOa). 

The firing rate is essentially the inverse of the time 
between successive waves passing through a given point. 
Thus, one may expect that decreasing the wave speed by 
increasing td would reduce the mean firing rate. This 
is, however, not the case; instead the firing rate is quite 
insensitive to the wave speed as shown in FigJSb. The 
reason for this is apparent in Fig[3^,b. It shows two 
simulations for the same network configuration but with 
different delay times td- For larger delay td additional 
waves are excited by the short-cuts as can be seen, for in- 
stance, at t — 380t£> where a new wave can be spawned at 
neuron 1071 for td —0.1 but not for td — 0.05 (marked 
by circles). This increases the firing rate, and in the case 
shown it even over-compensates the reduced wave speed, 
as can be seen by the higher density of waves for td = 0.1 
at larger times. 

For low p only a few short-cuts are present and many 
pathways leading to persistent activity consist only of 
large closed loops, implying low firing rates. As p in- 
creases the typical loop sizes decrease and network con- 
figurations with large minimal loops and correspondingly 
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FIG. 3: The firing rate is close to The firing rate 

averaged over 1000 configurations (5,000 for p > 0.2) as a 
function of the density of shortcuts p for different values of the 
delay td (N = 1000) (a) and for different values of the system 
size N (td = 0.1) (b). The theoretical maximal valueof 
is indicated by the dashed line. The bars indicate twice the 
standard deviation across configurations. 



low firing rates become increasingly unlikely. This is seen 
in the distribution function for the firing rate shown in 
FigEl As the distribution function is shifted towards 
larger firing rates with increasing p it narrows substan- 
tially reflecting the maximal firing rate set by the recov- 
ery time Tjj , which is marked by a dashed line in Figj5] 

The activity into which the network eventually settles 
is periodic in time and with increasing number of short- 
cuts the amplitude of the oscillations increases. This is 
shown quantitatively in Fig[51 which characterizes the 
magnitude of the oscillations by the standard deviation 
of the firing rate averaged over a large number of per- 
sistent configurations. As is apparent from FigJ^l the 
time for these oscillations to establish themselves from 
the excitation of a single neuron decreases with increas- 
ing p. This corresponds to the main result of [H, HH 
where it was found that in the small-world regime a 
network of Hodgkin-Huxley neurons and of FitzHugh- 
Nagumo neurons is entrained much more quickly by a 
cluster of driven, oscillating neurons. 

For larger values of p the activity patterns can be quite 
complicated. While in this regime all neurons get excited 
during an oscillation cycle, not all neurons and connec- 
tions between them are necessary for the persistence of 
the activity. This is illustrated in Figj7l The full raster 
plot is depicted in the top panel (black dots). Super- 
imposed on it are those neurons that are essential for 
carrying on the activity; they constitute the pathway of 




FIG. 4: For longer delay in an identical network, more waves 
can travel through the system. Raster plots for td = 0.05 and 
td = 0.1 with p = 0.1 and N = 2000. Circles mark where a 
new wave is spawned for td =0.1 but not for td = 0.05. 
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FIG. 5: Distribution of firing rates across configurations for 
various values of p. As p increases the distributions narrow 
in width and approach the value 1/Tjj (dashed line). N = 
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this activity [221] . It is is determined as follows. Once the 
simulation has been run, all the neurons that fire at the 
final time are labeled. For each of the labeled neurons, 
a search is carried out for presynaptic active neurons, 
i.e. neurons that fired one delay td ago and triggered 
the currently firing neurons. These 'causal' presynap- 
tic neurons are labeled in turn. This process is carried 
on backwards in time until t = 0. The pattern of la- 
beled neurons quickly converges to a small subset, see 
middle panel of Fig(7] Only the neurons in this much 
simpler subset contribute to the persistence of the pat- 
tern, i.e. the remaining neurons could be cut out of the 
network without destroying the persistent activity. In 
FigH the pathway of activity (shown blown up in the 
bottom panel) is periodic in time with a period that is 
only slightly longer than the recover time , indicated 
by the dashed lines. Such pathways have also been iden- 
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FIG. 6: Oscillation amplitude increases with the number of 
short-cuts. Dependence of the standard deviation of the fir- 
ing rate (for persisting activities) averaged over up to 5,000 
configurations for to = 0.1. TV = 1000. 
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FIG. 7: An example of a 'pathway' of excitation. The path- 
way responsible for persistence is shown in red and is longer 
than the recovery time Tjj , indicated by the dashed lines in 
the lower panel, td = 0.1 and p = 0.1. See text for details. 



tified in an experimental study of the photo-sensitive, 
excitable Belousov-Zhabotinsky reaction in which short- 
cuts were introduced through local optical excitation [2l[ . 

A striking feature of this regime is the fact that for a 
given network configuration there is an extraordinarily 
large number of different attractors. To assess the de- 
gree of multiplicity of attractors we focus on the regime 
with low short-cut density where all solutions are pe- 
riodic and characterize each solution by its period, its 
mean firing rate, and the standard deviation of the firing 
rate. Clearly, distinguishing different solutions by only 
these three measures may underestimate the total num- 
ber of attractors. Fig[5]illustrates the attractor multiplic- 
ity for a single, randomly chosen network configuration of 
N = 1000 neurons for p = 0.05 and t d = 0.1. In FigEJi 
presents each of the 471 different solutions that can be 
reached by exciting initially a single neuron with all other 
neurons being at rest. As shown more clearly in FigJSjD, 
which gives the number of initial conditions that lead to 
a solution with a given period (irrespective of their firing 
rate and its standard deviation), only few values for the 
period arise (note the logarithmic scale). In contrast, the 
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FIG. 8: Large number of attractors. a) Mean firing rate, its 
standard deviation, and period of all 471 attractors obtained 
for a single configuration for N = 1000, p = 0.05, and td = 
0.1. b) Number of initial conditions leading to an attractor 
with a given period. Many attractors, although distinct, have 
the same period, (note logarithmic scale), c) Number of 
initial conditions leading to an attractor with a given standard 
deviation of the firing rate. 



standard deviation of the firing rate (Figj8j;) can take on 
many different values, each representing a different tem- 
poral evolution of the firing during the period. 

The large number of attractors is typical for these net- 
works as seen in Fig|9l which presents the number of at- 
tractors as a function of the network size averaged over 20 
different configurations (black circles). A large fraction 
of the attractors has a quite small basin of attraction: 
within the restricted set of initial conditions in which 
only a single neuron is excited, they are reached from 
only a single initial condition (red squres). The overall 
number of attractors seems to increase roughly linearly 
with system size for larger values of N, while the num- 
ber of attractors with different periods (green diamonds) 
seems to grow more rapidly than that. Since the dura- 
tion of the transients grows with system size the com- 
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FIG. 9: Dependence on the system size. All attractors (cir- 
cles), attractors with basin of attraction 1 (squares), attrac- 
tors with different period (diamonds). 



putation time grows faster than N 2 . This precludes us 
from going to significantly larger system sizes than shown 
in Fig[9] [32}, which would be necessary to get a reliable 
estimate for this scaling. Preliminary computations indi- 
cate that allowing more general initial conditions signif- 
icantly increases the number of attractors beyond those 
shown in Fig|8] Thus, while the restricted initial condi- 
tions employed in FiglQ] suggest an almost linear increase 
in the number of attractors with system size, the full 
number of attractors may grow substantially faster. At 
this point the origin for the exceedingly large number 
of different attractors is not clear. In all-to-all coupled 
oscillator systems factorially large numbers of attractors 
have been found due to the permutation symmetry of 
that global coupling (23|. The small- world networks in- 
vestigated here do not possess any such symmetries. 

As the value of p increases, the distance traveled by 
the waves before encountering a long-range connection 
decreases. Consequently, the waves of excitation spread 
throughout the network more rapidly. This effect can 
be seen clearly in the progression of spatio-temporal pat- 
terns in Fij^a-e) . For sufficiently large values of p the 
activity spreads too quickly to sustain the activity, see 
Figj2j. Since the long-range connections are made ran- 
domly, the dynamics vary across network configurations. 
Thus, while the overall likelihood of persistent activity 
should decrease with increasing p, the actual network dy- 
namics depend on the particular network configuration. 

The dynamical mechanism leading to the extinction of 
network activity is easily elucidated [l6[. Once a neuron 
has emitted a spike, its voltage is reset, here to 0. As 
the voltage of the neuron recovers to its resting poten- 
tial, here equal to Voo, the activity spreads through the 
network, eventually finding its way back. Once this oc- 



curs, the neuron receives synaptic input equal to g ayn . 
This input will be only sufficient to trigger a spike in 
this neuron if the neuron has sufficiently recovered. It is 
therefore clear that the more rapidly the activity spreads, 
the less likely the network will be to exhibit persistence. 
Whether or not this mechanism leads to the extinction 
of activity in a given network depends on the particu- 
lar network topology. For fixed p many configurations of 
the network can be simulated to calculate the percentage 
of network configurations for which the activity is extin- 
guished, as shown in the left inset of FiglTUl In agreement 
with our intuitive argument, the likelihood of the activ- 
ity failing for a particular configuration drawn at random 
increases with increasing p. In fact, for large enough sys- 
tems it seems that there is a transition or, more precisely, 
a cross-over from a regime in which all network config- 
urations exhibit persistent activity, to one in which the 
activity will always fail. The transition moves to larger 
values of p as the s yste m size N increases. 

As discussed in this transition can be captured 
in a mean-field approximation in which the return time 
is identical for all neurons. In this case the dependence 
on the long-range component of the topology can be ex- 
pressed as a function of p alone. Setting the maximum 
return time Ta (the time needed for the activity to tra- 
verse the entire network) equal to the recovery time yields 
an upper bound for the density of long-range connections 
at which a transition from persistent activity to failure 
occurs 



Ta(Pc 



T 



(i) 



(5) 



An approximate form for Ta is easily derived. We as- 
sume that a single neuron fires at time t = 0. Given a 
density p of shortcuts, we expect a long-range connection 
will be reached after l/p neurons have fired, which occurs 
after a time td /2p since there are two wave fronts. Four 
wave fronts are now present and so 2/p neurons will fire 
in a time T^jlp at which point on average two new wave 
fronts are generated, and so on. During the k th cycle, 
the number of neurons excited is 2 k ~ 1 /p. After n cycles 
all the neurons have been excited, which results in 



J2^ k =pN 



and leads to a time 



Ta( P ) 



k=0 



TD 
t 

2p 



TD- 



ln(l +pN) 
2pln2 



(6) 



(7) 



Equation is a purely geometric result, since the 
time it takes for activity to traverse the entire extent of 
the network is related trivially to the largest distance 
in the network. The geometric mean-field properties 
of small- world networks have been worked out by New- 
mann, Moore and Watts. In [24[ they calculate the frac- 
tion of a small-world network covered by starting at a 
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single point and extending outwards a distance r in both 
directions (equivalent to waves having spread for a time 
rro in our case) in a continuum limit. In [24| they refine 
the continuum limit calculation by taking into account 
two effects that were omitted from their earlier calcula- 
tion [25| and from ours. Firstly as they trace out the 
network, a jump via a long-range connection may reach 
a part of the network that has already been traced over 
and should therefore not be counted. We interpret this 
as activity spreading via long-range connections, which 
is injected into a neuron that has already fired but is not 
ready yet. Such a connection would therefore be ineffec- 
tual. Secondly, when two 'traced-out' sections meet, they 
stop and no longer contribute. We interpret this as wave- 
fronts which meet and annihilate and thus, thereafter, do 
not contribute. Considering these two additional mech- 
anisms, they develop a two-component model that de- 
scribes, effectively, the fraction of the network covered 
and the number of fronts. The result, interpreted within 
the context of equation ([T]), yields, 



^ tanh 

pNJ 



4 \ P T A (p) 
pNJ 2t d 



(8) 



Setting T A (pcr) = T. 



R ' 



yields the density p^f FT ^ at the 

failure transition. 

The main panel in FigfTU] shows the failure rates, 
rescaled by the critical density p^f FT ^ calculated from 
Eq.©. All failure curves intersect at the same value of 
p, which defines therefore the transition point p cr . Con- 
sistent with the upper bound obtained in the mean field 
theory p cr < p~Jf FT ^ . Unfortunately, we need to point 
out that the good quantitative agreement reported in [l6j 
was spurious; there we incorrectly used Tr instead olT R x ^ 
as the recovery time in (J5J) (cf . Fig fTTj) . For comparison 
with the approximate mean field calculation the right in- 
set shows the failure curves rescaled by p cr as obtained 
from ([7]). 

The central quantity determining the persistence or 
failure for a given short-cut configuration is the recovery 
time T^K It is worth pointing out that this time is not 
the same as an absolute refractory period T r of the neu- 
ron. While T r is a property of the neuron independent 

of its neighbors, T R depends strongly on the strength of 
the coupling between the neurons. As long as the refrac- 
tory period is shorter than the recovery time it has little 
effect on the persistence of the network. This is illus- 
trated in Fig llll which shows the fraction of persisting 
configurations for a network of neurons with an absolute 
refractory period T r during which they do not respond 
to any input from other neurons. During that time their 
membrane voltage V , however, still relaxes towards its 
resting value. For T r < 2td the second input character- 
istic of propagating waves (cf. ([!])) is unaffected and the 

relevant recovery time is T R \ The resulting failure curve 
(T r = 0) corresponds to those in FigJTni For 2td < T r 
the second input is suppressed. In this low-p regime the 
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FIG. 10: Failure Transition as a function of system size N. 
Fraction of failing configurations rescaled by p cr from Eq(H] 
Left inset: unsealed data. Right inset: data rescaled by p cr 
from EqFTl 
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FIG. 11: Dependence of failure on refractory period T r 
N = 1000, 7"d =0.1 {Tr = 2.83), and 500 configurations. 
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persistent states do not depend on neurons receiving any 
further additional inputs before their firing (cf. SeclIV|) 
and the failure transition is independent of T r as long as 
2t d < T r < T R . Only for T r > T R does the transition 
depend on the absolute refractory period. 



IV. THE DISORDERED REGIME: CHAOTIC 
TRANSIENTS FOR SLOW WAVES 

For small values of p we have seen that the likelihood 
of persistent activity is high. In this regime, the spatio- 
temporal activity, despite the complex, heterogeneous 
topology of the network itself, is highly regular and most 
often periodic. As p increases a transition in the like- 
lihood of persistent activity occurs and more and more 
network configurations exhibit activity which peaks and 
then shuts down. Interestingly, for large enough td there 
is an additional change in the behavior of the network as 
p increases, see FigfTS] Specifically, one finds that while 
the likelihood of failure initially increases for low p, as de- 
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Density of short-cuts p 



FIG. 12: Failure curves for different values of the delay td — 
0.06,0.08,0.1,0.12,0.14,0.16. Each symbol is the average of 
2000 configurations. For large enough delay, td = 0.16 and 
0.18, the likelihood of failure before a finite time T* = 100 
becomes non-monotonic in p. 
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FIG. 13: Raster plots and firing rate for four typical configu- 
rations for 'slow' waves, td = 0.16. p = 0.01,0.2,0.4 and 1.0 
for (a) through (d) respectively. The activity in the 'high' p 
regime seen in (d) is unlike that seen in the other three. It is 
noisy and exhibits synchronized population spikes. 



scribed in the previous section, it then turns downward 
again for larger values of p, see the curves for tjj = 0.16 
and 0.18 in Fig[T2l To understand why this is so, we 
first describe the spatio-temporal dynamics underlying 
this seemingly persistent activity. 

FigCS] shows four typical raster plots of the activity 
seen for td = 0.16 as a function of p. For values of p 
below or near the theoretical transition to failure (panels 
(a) through (c)), the activity is similar to that seen in 
FigEl In the regime beyond the maximum of the fail- 
ure curve (panel (d)) the activity is chaotic (cf. FigfT4k 
below) and exhibits irregular population spikes reflect- 
ing near-synchronous activity involving a large fraction 
of the neurons. 

A more detailed, quantitative analysis for td = 0.18 
shows that the change in behavior occurs already be- 




Density of Short-Cuts p 

FIG. 14: Oscillation amplitude depends non-monotonically 
on the density of short-cuts. Mean and standard deviation 
of the firing rate is shown for those configurations (out of 
200) for which the activity persists for at least 15,000 steps. 
(td = 0.18, N = 1000). 

fore the maximum of the failure curve. For the fast 
waves (small td) the oscillation amplitude as measured 
by the standard deviation of the firing rate increases 
monotonically with p (cf. Fig[6]). For slower waves it 
is, however, non-monotonic and decreases over the range 
0.4 < p < 0.6 (FigHUrc = 0.18). More instructive yet is 
the dependence of the spectral entropy of the firing rate, 

£ = -]TSMmSM, (9) 

which measures the number of significant peaks in the 
power spectrum S(u>). Its average over 200 configura- 
tions rises significantly in this range of p, as shown in 
Fig. 1151 indicating an increase in the complexity of the 
dynamics. The variability of £ across the configurations 
exhibits a (broad) maximum in the transition region and 
reaches very small non-zero values around £ ~ 0.06 in 
the strongly chaotic regime. We have not investigated 
details of this evolution towards chaotic dynamics, which 
depends on the individual configurations of the short- 
cuts. 

What causes the dynamics to re-emerge as p increases? 
The answer lies in the interplay between network topol- 
ogy and the delay tjj. In order to develop the mean- 
field model, eq.© we assumed that the firing of each 
neuron is limited by the recovery time However, 
even for p < 1 the construction of the network allows 
for some neurons to receive more than one incoming 
short-cut. For our network it can be shown [22j that 
for p small, the fraction of neurons with two incoming 
short-cuts is S2 ~ p 2 /2, which is indeed negligible (33|. 
Thus, the mean-field results should hold if the transi- 
tion occurs at sufficiently low values of p. However, for 
\p — 1 1 -c 1 the fraction of neurons with two inputs be- 
comes S2 = 1 — a/2/2 + (p — l)/2 which is nearly 0.3 for 
p = 1. Thus, a significant subpopulation of neurons likely 
receives several inputs per cycle. Such neurons would 
not be constrained by the recovery time T^' but rather 
would be primed to fire earlier, potentially allowing the 
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FIG. 15: Temporal behavior becomes increasingly complex 
with increasing p. a) Representative power spectra for p — 
0.2 (£ = 4.6) and p = 1.0 (£ = 7.4). b) Spectral entropy 
S (mean and standard deviation across 200 configurations). 
(td = 0.18, N = 1000 and 15,000 time steps) 



activity to persist where it otherwise would fail. We can 
easily calculate the recovery time for a neuron i which, 
by virtue of several incoming connections, has received n 
inputs since its last firing 



T^\t 1 ,...,t n ) = \n 



V» 9syn y^ r — 1 6 



V« 



Qsyn 



l 



(10) 



Note that Eq.fTU]) reduces to Eq.© for n = and to 

Eq0]for n = 1 with ii = 2td. In general, T^™- 1 depends 
on the n firing times of those neurons providing input to 
neuron i, which are unknown. However, since integrate- 
and-fire neurons become increasingly sensitive to their 
input as time passes after their firing, the value of Tn(n) 
in Eq. (fTU]) is bounded below by Tjj"^ in , which occurs 

(n) 

when all n inputs coincide at min itself. This time is 
given by 



T, 



(») 



R.min 



Vex 



Vx 



TICJsyi 



- 1 



(11) 



For small enough p the inter-spike intervals (ISI) of al- 
most all the neurons are bounded below by \ which 
allows us to calculate the time for activity to spread 
throughout the whole network using a geometrical ap- 
proach. For higher values of p there may be a subset of 
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FIG. 16: Distribution of interspike intervals (ISI) for td = 
0.16 and a range of p. For low p all ISI are above the recov- 
ery period T^ 1 ' (dashed line). For p > 0.6 increasingly more 
multiple inputs allow shorter ISI. 



neurons with shorter allowable ISI. However, many neu- 
rons will still only receive a single input per cycle and 
their activity should reflect this fact. This is borne out 
in the distribution function for the ISI shown in FigJTBJ 
'Fast' spikes with ISI < occur appreciably only for 
p > 0.6 and increase further in frequency with increas- 
ing short-cut density. Our previous analysis [l6l ] showed 
that the spikes with ISI > occur in population bursts 
with no such spikes in between. In the absence of other 
spikes the activity would die out during these periods. 
However, due to the large number of short-cuts there is 
a substantial number of neurons that have received mul- 
tiple inputs and are consequently primed to carry over 
the activity to the next cycle, thereby allowing the 'slow' 
spiking neurons to recover. 

Increasing the delay td contributes in a number of 
ways towards bridging low-activity periods. For larger td 
the failure transition is shifted towards larger short-cut 
densities enhancing the number of multiple-input neu- 
rons significantly. At the same time, in order to bridge 
the time between the relatively short return time Ta and 
the recovery time of the slow neurons fewer fast neu- 
rons with multiple input are needed if the delay is longer. 
Moreover, for the integrate-and-fire neurons © lat er in- 
puts have a stronger impact on the recovery period than 
earlier ones (cf. Eq. pH)) ). With increasing td all inputs 
are shifted to later times relative to the last spike of the 
respective neuron, which reduces the recovery time sig- 
nificantly. The increased delay is, however, not necessary 
to reach the regime of prolonged activity. As shown in 
FigHZl for fixed delay td = 1.4, increasing the system 
size from N = 1,000 to N = 16,000 shifts the failure 
transition to sufficiently large p that the number of neu- 
rons with multiple inputs is sufficient to bridge the gap 
in slow-neuron activity even for this lower delay time. 

For low values of p the spatio-temporal dynamics are 
most often periodic. In those cases the dynamics can 
truly be called persistent. For p ~ 1, the chaotic na- 
ture of the dynamics precludes such a clear assessment 
and, in fact, failure is possible even after very long times. 
Since persistent activity relies on a bridging of the qui- 
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FIG. 17: Prolonged activity in large systems. Failure rates 
up to a final time T* — 28 for td = 0.14 (400 configurations). 
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FIG. 18: The firing rate for four values of the delay td, with 
p — 1. The network architecture and initial condition are 
identical for all four simulations, only the delay has been 
changed. Failure tends to occur at longer times for larger 
delays. 



escent period by a string of neurons that have multiple 
incoming short-cuts it is necessary that these short-cuts 
are actually activated during the previous cycle and at 
suitable times. Thus, while in one cycle the activity dur- 
ing the burst may have been able to excite such a chain 
the different activity pattern in the next cycle may fail 
to do so and the activity could die out. Indeed, we find 
that for these large short-cut densities the activity even- 
tually fails for essentially all configurations. Examples 
of such long-lived transients are shown in Fig |181 where 
firing rates are shown for the same configuration of short- 
cuts but increasing values of To- Clearly, the delay has 
a strong influence on the duration of the transient (note 
the change in scale on the x-axis). 

While overall there is a trend for the lifetimes of the 
transient activity to increase with increasing delay, the 
actual dependence on to is more subtle. FigfT^lshows, as 
a function of to , the fraction of configurations for which 
the activity fails before the final time T* is reached. The 
dependence on To exhibits an amazing degree of struc- 
ture. Most surprising is the finding that an increase in to 
does not always decrease the number of failures, but can 
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FIG. 19: a) Failure rates for a range of final times T* = 5Tr, 
WTr, 20T r , 40T r , IOOTr based on 2000 configurations with 
p — 1 and TV = 1, 000. The range covered in b) is marked by 
a box. b) Fine structure in the failure rate for T* = WOTr 
based on four runs with 8000 configurations each. Black line 
gives average over the 4 runs. 



in fact enhance the probability of failure. These changes 
can occur over very small intervals in To as shown more 
clearly in Fig |19b . This fine structure is reminiscent of 
'resonances' or 'windows' in which the delay is To 'opti- 
mal'. While details of the mechanism underlying this 
structure are not understood yet, it is clear that the 
dependence on to reflects the significance of the ratio 
td/Tr. This is further illustrated in Fig|2"01 where the 
failure rates are shown for a reduced number of neurons, 
/V = 500. While the failure rates are higher overall in the 
smaller system, the locations of the windows in To are 
not substantially altered. If however, the recovery time 
is reduced from Tr — 2.83 to Tr — 2.79 by increasing the 
synaptic strength from g = 0.2 to g — 0.202, the windows 
are clearly shifted to lower values of to ■ 

To assess whether any significant fraction of the con- 
figurations leads to truly persistent activity we consider 
the failure fraction J 7 as a function of the final time T*, 
with the aim to extrapolate to T* — > oo. Fig[3T] shows 
the fraction of persistent configurations, 1 — T(T*), for 
a range of delays to on a logarithmic scale. As may 
have been aniticipated from the non-monotonicity seen 
in FigQjJJ the fraction of failing configurations is largest 
for to — 0.167 over the whole range of times monitored 



12 



a 
o 
'& 

u 

•- 

fa 

Sn 
3 

'£ 
fa 



0.8 
0.6 
0.4 
0.2 



ft 



•-• T =20T R 




•-• T*=100T R 




O— O N=1,000 T* 


=100T R 


m-m g=o.202 I 


=100T R 




13 0.14 0.15 0.16 0.17 0.18 
Delay x D 



a 

3 

cr 
u 

fa 

i 
1 



10000 



1000 



100 




100 200 
Failure Time T 



FIG. 20: Failure rates based on 8000 configurations with p — 1 
for N = 500 for g — 0.2 and g — 0.202. For comparison also 
the case N = 1000 is shown (cf. Fig[HJ. 



FIG. 22: The distribution of characteristic failure times across 
50,000 configurations. The distribution decays exponentially 
for long lifetimes. N = 200, p = 1, td = 0.16 
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FIG. 21: Complementary failure fraction 1 — T as a function 
of time for a range of delays td . The decay is non-exponential. 
Inset: Failure fraction T for the same data. 



and a very rapid drop in the failure rate is seen when 
going from tjj = 0.17 to T£> = 0.18. Considering the 
long-term behavior it is apparent from Fig[T!)] that the 
decay is non-exponential. 

To obtain an approximate, analytic form for the failure 
fraction T(T*) we consider first - for a fixed value of p 
- a single fixed configuration. The duration of the activ- 
ity before failure will then depend on the specific initial 
condition chosen. In these simulations we choose 2,000 
random initial conditions V = Vq + £ with Vq — 0.85 
and —0.5 < £ < 0.5 picked from a uniform distribution. 
The resulting distribution for the failure times exhibits 
exponential behavior for large times. This allows the 
extraction of a characteristic failure time T associated 
with this configuration. To reduce the computational ef- 
fort these simulations are done in a smaller system with 
N = 200. The exponential distribution suggests that the 
chaotic dynamics effectively lead to a fixed probability 
for the activity to die out after each population spike. 

Across all configurations this leads then to a distri- 
bution p(T) of characteristic failure times, in terms of 
which - averaged over many configurations - the fraction 



of failures up to a time T* is given by 

r°° * 
T{T*)= dTp{T){\- f3e-^). (12) 
Jo 

Extracting the characteristic lifetimes T of 50,000 con- 
figurations we find that the distribution p(T) can also be 
well fit by an exponential function for long lifetimes, 

lim p(T) cx e~ aT . (13) 

T — >oo 

Fig [2^1 shows the result for td = 0.16 and p = 1 (N = 
200). Inserting the asymptotic behavior of Eq. (fT3|) into 
Eq. (fT2|) yields for large t 

T{t) = 1 - 2Py/atK 1 (2Vai) (14) 

where Ki (x) is the modified Bessel function of the sec- 
ond kind of first order. For long times Eq03] can be 
expressed as 

T{t) ~ 1 - /V^(ai) 1/4 e- 2 ( ttt > 1/2 (15) 

displaying stretched exponential behavior. As Fig|2"3l 
shows, Eq[14| provides quite a good fit to the time de- 
pendence of the failure fraction. For to — 0.18 the fit 
corresponding to a = 0.67 • 10~ 5 and f3 = 0.91 is good 
over essentially the whole range of times. For td = 0.165 
(inset) the fit is not quite as good; its curvature seems 
smaller than that of the data. Starting at t = 3,400 one 
obtains a = 0.12 • 10~ 3 and (3 = 0.64. This fit overshoots 
the data for large times (blue dotted line in Fig |2"3"|) . The 
fit with a = 0.10- 10~ 3 and (3 = 0.54 (red dashed line) re- 
duces the overshoot but is not as good for smaller times. 
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FIG. 23: Complementary failure fraction 1 — T for p = 1 and 
td ~ 0.18 (inset for td = 0.165). The dashed red and dotted 
blue lines show fits to Eq. (fT4)l . 



V. CONCLUSION 

In this paper we have used a minimal model to study 
the influence of the network topology on the dynamics of 
excitable elements. The network consisted of local con- 
nections between neighboring elements on a ring that are 
supplemented by randomly chosen short-cuts connecting 
distant elements. Having neural systems in mind we have 
taken the short-cuts to provide uni-directional connec- 
tions rather than the bi-directional ones that would arise 
in epidemic contexts [l5| or in regular diffusive processes. 

Depending on the density of short-cuts and the speed 
of the waves propagating through this network we find 
three regimes. For low, but non-zero density of short-cuts 
the activity persists for essentially all network configura- 
tions when starting from an initial excitation of a single 
neuron. The activity is predominantly periodic and the 
mean firing rate of these states shows only little depen- 
dence on the wave speed or the density of short-cuts once 
p > 0.05 and it is reasonably close to the maximal fir- 
ing rate allowed by the recovery period of the neurons. 
The recovery period is, however, not to be confused with 
an absolute refractory period; rather, it is the time after 
which synaptic input of the strength used in the compu- 
tations is sufficient to trigger a new spike. This recovery 
period can be much longer than the usual absolute re- 
fractory period, for instance, if the neurons exhibit slow 
after-hyperpolarization as it underlies the slow oscilla- 
tions (< 1Hz) observed in vivo in cat [IBl and in cor- 



tical slices of ferret 27]. There the relative refractory 



period induced by the after-hyperpolarization can last as 
long as a few seconds. The dependence of the propaga- 
tion speed of such slow oscillations on the connectivity 
has been studied in cortical models [H, [29[. No true 
short-cuts were employed, instead the spatial width of 
the Gaussian giving the probability that two neurons are 
connected was varied [291 ] or a tri-modal probability dis- 
tribution was used to capture a certain patchy connec- 



tivity in cortex [28|]. As expected, the speed was found 
to increase with the width and it was conjectured that 
this connectivity dependence is the origin for the large 
difference observed for the slow waves in olfactory cortex 
and neocortex, respectively [29| . 

Even though for small numbers of short-cuts the long- 
time dynamics are periodic, the over-all behavior of the 
system can be quite complex due to the large number of 
stably coexisting solutions for a given network configu- 
ration. Whether the number of attractors grows as fast 
with the system size as in globally coupled oscillator net- 
works, where the permutation symmetry essentially leads 
to a factorial growth of the number of attractors and to 
attractor crowding [23], is not known at this point. In 
fact, so far we have not been able to reach saturation 
of the number of attractors with increasing number of 
different initial conditions and the mechanism underly- 
ing this large number of attractors is not apparent yet. 
It is clear, however, that noise will induce a persistent 
switching between these different attractors [221 ]. 

As the density is increased the number of network 
configurations that allow such a persistent activity de- 
creases until essentially no persistent activity is possible 
anymore. For fast waves the transients after a localized 
excitation consist of a single population burst. For slower 
waves, however, for which the cross-over to complete fail- 
ure occurs at yet larger short-cut densities, the transients 
can be exceedingly long comprising thousands of popu- 
lation bursts. The times at which the failures in activ- 
ity averaged over different configurations occur are dis- 
tributed according to a stretched exponential. It arises 
from the exponential distribution of the lifetimes char- 
acterizing each configuration. The mechanism responsi- 
ble for these long transients presumably differs from that 
operating in diluted random networks of pulse-coupled 
oscillators [3(ij . 

While the fraction of failing configurations up to a 
given time follows a decreasing overall trend with de- 
creasing wave speed, it exhibits intricate fine structure 
that includes even sharp, resonance-like increases of the 
failure fraction with decreasing wave speeds. This is sur- 
prising, since naively one might expect that a decrease in 
wave speed would allow additional, shorter loops to con- 
tribute to the activity and therefore enhance the chances 
for persistence. However, the result indicates that the ac- 
tivation of one loop can, in fact, make other, previously 
active loops impossible and thus induce failure. While 
it is clear that such a switching between loops can occur 
with increasing td , this mechanism is not yet understood 
in any detail. 

The cross-over to failure, which can be understood an- 
alytically in detail based on a mean-field theory for the 
size of these idealized small-world networks, provides the 
basis for understanding a number of recent investigations 
employing more complex neural network models fla . [l^ ] . 

In [l8| the connection between network connectivity 
and epilepsy in hippocampus was investigated by consid- 
ering noisy and more elaborate versions of our model. 
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It was found that for low numbers of short-cuts the 
noise drives only low-level activity, which was associated 
with normal behavior. With increasing short-cut density 
the activity strongly increased due to the recruitment of 
many more neurons by a noise-driven event. This activity 
was likened to seizing activity. This regime corresponds 
to the connectivities supporting persistent activity. Yet 
further increases in the short-cut density were found to 
induce bursting dynamics in which irregular bursts that 
involved a large fraction of all neurons are separated by 
periods of quiet. One would expect such a behavior for 
connectivities in the failing regime in which each noise- 
triggered event leads to a single population spike that 
brings essentially all neurons into the recovery period. 

In [l^| the system was driven by a set of localized pace- 
maker neurons. It was found that the short-cuts can lead 
over a number of driving cycles to the build-up of bursts 



during which a large fraction of neurons fire within a 
small time window. The time to build up such bursts 
and the time between them was found to decrease with 
short-cut density. Again, the appearance of such bursts 
is related to the failing configurations in systems without 
driving. As expected from this observation the bursting 
behavior was supplanted by persistent activity when the 
wave speed was reduced (cf. Figri~2"j). The slow build- 
up towards the burst, however, is relatively specific to 
the Morris-Lecar model employed in that study for the 
individual neurons [19]. 
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spike could represent burst then a single 'spike' would 
reasonably lead to excitation and the bckground activity 
would indeed be much lower (cf Butera) 

is in the figure with global inhibition and noise Iq really 
0.1? 

In the input file for noise I have listed 10=0.9. 

==================== superbursts (cf. 

WaNa06) : in our case bursts always cease because 
of subtle lack of bridging . therefore expect not pre- 
served duration of bursts in the presence of noise. They 
probably have some slow time scale that cuts off the 
bursts. similar is the rapid spreading through the network. 



I. INTRODUCTION 



s:intro 



It has been widely recognized that the connectivity of 
a network has profound impact on its function. Substan- 
tial efforts have been devoted to the characterization of 
the network connectivity [? ] and on its influence on 
the synchronization of oscillators on such networks [? ] . 
Much less studied is the effect of the connectivity on cou- 
pled excitable elements, which includes models of certain 
types of epidemics [? ] . 

An essential feature of the brain is that its neurons 
are connected in a higly non-trivial way; often neurons 
receive input both from their near-by neighbors as well 
as from quite distant neurons. Moreover, a large fraction 
of the neurons responds in an excitable way to individ- 
ual inputs. It is therefore important to gain a thorough 
understanding of the influence of the connectivity of ex- 
citable neurons on their overall behavior. 

A very simple model for the study of the effect of cer- 
tain topological aspects of the connectivity on the overall 
dynamics consists of excitable elements coupled locally to 
their immediate neighbors as well as to a fraction of ran- 
domly chosen elements in the system. In previous work 
we have shown that even in this very idealized model, 
which is a slight modification of the original 'small- world' 
model introduced in [? ] , interesting behavior is found de- 
pending on the topology, including a cross-over from per- 
sistent activity to failuer of propagatin and exceedingly 
long chaotic transients [? ]. Morevoer, through this ex- 
treme idealization one can even obtain analytical results 
for this cross-over. Extended versions of this model have 



been studied in somewhat more concrete biological con- 
texts. In [? ] the connection between topology and the 
tendency of the neural network towards epileptic seizures 
has been studied. In [? ] bursting behavior found in the 
pre-B6tzinger complex in the ??? has been connected 
with the topology of the network. Both scenarios can be 
interpeted quite well based on an understanding of the 
failure cross-over in the basic model of [? ]. 

Here we present a more detailed discussion of the dy- 
namics of a small-world network of excitable elements. 
In Section ?? we define the model which consists of 
'Integrate-and-Fire' neurons that are coupled via exci- 
tatory pulses, which correspond to very brief current in- 
jections. In ?? we give a more detailed characterization 
of the cross-over from persistent activity to failure for 
the case of relatively rapid propagation of the excitation. 
In Section ?? provide a detailed analysis of the exceed- 
ingly long chaotic transients in the regime of relatively 
slow propagation. In Section ?? we discuss our results in 
light of other work on neural networks with small-world 
topology [? ? ] and experiments on cultured networks 
of neurson [? ] . 



II. THE MODEL AND NETWORK TOPOLOGY 



s:model 



We consider a one-dimensional network of N, identical 
integrate-and-fire neurons 



N 



T-^j- = -Vi + Tg syn ^2 WijS(t - t° - T D ) + Tlext, JF_ (1 
V(t+) = Vres 



whenever 



V(t~) = Vth, res@) 



where r is the membrane time constant, g syn is the 
synaptic strength, tu,j = or 1 indicates the absence or 
presence respectively of a synaptic connection from neu- 
ron j to neuron i, P is the time at which neuron j fires 
an action potential, td is the delay in the neuronal in- 
teraction, and I ex t is an external current. Post-synaptic 
currents due to recurrent synaptic activation are consid- 
ered instantaneous and are therefore modeled as Dirac- 
delta functions. Eq.(??) reflects the fact that whenever 
the voltage of neuron i reaches the threshold value Vth i a 
spike is emitted and the voltage is reset to the value V res . 
In this work we take t = 1, V res — and Vth — 1 without 
loss of generality. We also impose the constraint that the 
neurons be excitable, i.e. I ext < 1. For a study of the 
dynamics in a similar model with oscillatory neurons, i.e. 
here I ext > 1, see [? ? ]. 



Iq corresponds to spontaneous background activity, 
should have Poisson distribution, we ignore that random 
aspect. 

We choose a simplified cortical-like architecture for the 
network in which each neuron is connected to 2k neigh- 
bors, i.e. Wij — 1 for j < i ± k and j ^ i. In addition, 
pN random, unidirectional connections arc made. The 
parameter p thus gives the density of random connec- 
tions as a fraction of the number of neurons N. 

The dynamics arising in Eqs(??) and (??) for purely lo- 
cal connectivity (p — 0) depend, in the absence of noise, 
on the interplay between the strength of the synapses 
g syn and the number of neighbors 2k. This is most eas- 
ily illustrated for nearest-neighbor connectivity (k = 1). 
In this case, if the pre-synaptic input is insufficient to 
cause a spike, i.e. g syn + I ex t < 1, no network activity 
will occur whereas if g syn + Iext > 1, a wave of speed 
1/t d is generated. If g syn + I ext (l — e~ TD ) < 1 the wave 
propagates outward from the site of initiation as each 
neuron contributes exactly one spike, see Fig.??(a). If 
9syn + ^ext(l — e~ TD ) > 1 then the wave entrains all 
the neurons in its wake eventually leading to synchro- 
nization of the network (unless autapscs arc included 
the network breaks up into two synchronous groups of 
neurons that fire out of phase with one another), see 
Fig.??(b) In the general case, a wave can be sustained 

if g syn e TD J2 n =i e ~ nTD + text > 1- That is, a neuron re- 
ceives inputs from k neighbors as the wave approaches, 
with an input at a distance n discounted by e( 1- ™) TD . 
Unless otherwise noted we will focus on the case of near- 
est neighbor coupling (k = 1) for simplicity, and will 
consider the regime in which waves of excitation prop- 
agate, but do not entrain activity in their wake. This 
choice constrains the allowable values of I ex t and g syn - 
In particular, we take I ex t = 0.85 and g syn = 0.2 unless 
otherwise noted. 

Eqns(??) and (??) with the above-mentioned assump- 
tions are therefore a minimal model for the generation 
and propagation of waves in cortical-like tissue. As we 
shall see, the addition of random, long-range connections 
qualitatively alters the dynamics of the network, allowing 
for a rich variety of spatio-temporal patterns. 

HR td is now factor 10 smaller since no rescaling of 
time by membrane constant 



III. THE ORDERED REGIME: PERSISTENT 
ACTIVITY 



s:order 

Once the input current I ext and synaptic strength g syn 
have been fixed, the dynamics arising in Eqns(??) and 
(??) are determined by the remaining two parameters: 
the fraction of randomly placed connections p and the 
delay in the neuronal interaction tj> 

The dynamics given a finite value of p differ quali- 
tatively from those for p = 0. The presence of long- 
range connections allows the waves of excitation to be 
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FIG. 1: Dynamics in the flocal network of 50 neurons with 
nearest-neighbor coupling. Parameters are I ext — 0.85 and 
td = 0.1. (a) g syn = 0.2: An example of a propagating 
wave. The wave fronts meet and annihilate at time 2.5. Top: 
Raster plot showing the times at which the neurons fire an 
action potential. Middle: The averaged firing rate of the 
network. Bottom: The voltage of neuron number 20. (b) 
gsyn = 1.0: The wave entrains all the neurons in its path. 
The final state of the network is one in which two synchronous 
groups of neurons fire out of phase with one another. Note: 
The maximum possible firing rate is 1/td = 10. 

singlewave 



re-injected into portions of the network which have al- 
ready been excited. This process of re-injection may lead 
to persistent network activity as shown in Fig.??. As the 
waves spread outward from the initial site of stimulation 
they encounter long-range connections and are injected 
elsewhere in the network. Wavefronts that meet annihi- 
late. After some time, the activity settles into a stable 
pattern in which the rates of generation and annihilation 
of the waves balance. The resulting activity is periodic 
in time and with increasing number of short-cuts the am- 
plitude of oscillations increases. This is shown quantita- 
tively in Fig.??, which characterizes the magnitude of hte 
oscillations by the standard deviation of the firing rate 
averaged over a large number of persistent realizations. 
Also apparent from Fig.?? is the fact that the time for the 
oscillations to establish themselves from the excitation 
of a single neuron decreases with increasing p. This was 



